On the Implementation of Broadcast, Scatter and Gather in a Heterogeneous Architecture

This paper considers the implementation and evaluation of broadcast, scatter and gather communication operations in heterogeneous machines organized as a Hierarchy of Processor-And-Memory (HPAM). The top levels of the hierarchy consist of a small number of fast processors, whereas the bottom levels consist of a large number of slow processors. Each HPAM level consists of a homogeneous processor mesh. Routing within each level and across levels uses an extended form of X-Y dimension ordered routing. The execution times of three collective communication operations are analytically evaluated in the context of two-level HPAM machines. For each operation, two di erent alternatives are considered depending on the absence or presence of hardware support. For each alternative, an e cient implementation is characterized. Furthermore, the two alternatives are compared and the gain achieved by using hardware support for these operations is assessed.